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AMPHORE - A WORKBENCH FOR FILM DOCUMENTATION 
[AMPHORE-Ein Arbeitsplatz zur Filmdokumentation] 

AMPHORE - a movie documentation workbench 

AMPHORE is a client server system for the documentation of moving image material. 
The server mainly is formed by a full text database with SGML capabilities while the 
clients are PC working places equipped with software for documentation and retrieval 
of movies and/or movie parts. In AMPHORE the complete film material is provided 
in digital form and thus can be used for content-oriented documentation and retrieval 
in a convenient way. This enables the documentalist to build very detailed indexes 
allowing access by sequence or even by shot. The film descriptions are based upon a 
syntactical, thesaurus-controlled indexing which reflects the films* diverse action 
strings and levels. 

1 . Introduction 

Two trends seem to shape documentation work in the near future: the increasing 
automation of text indexing and the growing importance of nontextual information, particularly 
audio, images and moving images. The value-preserving and value-creating work of the 
documentarist will thus be aimed more at indexing nontextual material, which raises the issue of 
the documentary workbench of the future. Media archival work will have to satisfy new 
requirements as expressed particularly in the new services of archives that today have 
crystallized out at least to some extent. 

• Similarly to that which has long been the case for text archives (online databases), 
media archives will also be searchable via data networks. The Worldwide Web Internet service 
shows very clearly that such services are already coming into existence. Present network 
bandwidths permit the transmission of high-quality material only in exceptional cases, which is 
why for the near future such services will offer only "preview" quality, which is sufficient for 
sorting through material that has been discovered. Moreover, such a restriction of access to the 
original material may well be in the interest of the respective provider. 
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• For new productions (no matter of what type of media) it is economically and also often 
qualitatively advantageous to archivers and producers if more use can be made of archive 
holdings than previously. Such a demand increases the importance of content-oriented access to 
the holdings and additional typical production demands arise for research, such as the search for 
a film scene of a given length, camera setting, illumination, etc. Alongside the "classical" 
content-related and formal search criteria, therefore, new characteristics worthy of 
"documentation 1 " are arising and documenting them particularly requires direct access to the 
archive material. 

Precisely in regard to the content-related indexing of film material, today's film archives 
fail to differing degrees to meet these coming requirements on media documentation. This 
documentary gap in the film archives can be crudely classified by the respective personnel 
situation: 

• For archives with minimal personnel resources, content-related indexing takes second 
place to formal indexing. This is hardly likely to change in the near future, because not even a 
simple sorting of the material can be guaranteed in this case. 

• In film archives with small holdings there are generally already attempts at 
content-related indexing, but almost always the entire film or piece must serve here as the 
documentary unit. New acquisitions are examined, but a more detailed documentation, down to 
the level of scenes or even cuts, cannot be achieved in the time available because of the technical 
resources. 

• If there is content-related sequence indexing, this is done at a very high personnel cost, 
which in turn implies that only a few large institutions can afford this expense. 

The logical goal must therefore be to use new technologies such that a high precision of 
documentation can be achieved with a lesser expense than previously, 2 a goal which has been 
envisioned in the AMPHORE project presented in the present work. 

The AMPHORE (Audio-visual Media Platform for the Highlighting, Organization and 
Retrieval of Entities) project is based on an idea from C. Carlson [2], the implementation of 
which was started in fall 1994 on the basis of a cooperation between the IWF (Institute for 
Scientific Film) of Gottingen and the IPSI (Institute for Integrated Publication and Information 
Systems) of the GMD in Darmstadt. At the present time, a first prototype is available, already 
featuring the three central components of the overall system: 

• a database for managing film documentation 

• a (software) system for the documentary acquisition of films (see section 3) and 

• a search system (see section 4). 
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While the database is a commercial SGML-capable document management system, the 
acquisition and search systems are developments of IPSI. Thereby it is possible to feed prior 
conceptual considerations (see section 2) directly into the software development. 

2. Prior considerations and conception 

In current research, there are several approaches that promise progress in film 
documentation. Here are only the most important ones: 

1. The basis for any content-related documentation of films is a decomposition of the 
films into subsequences. The concept of sequence can hardly be defined exactly, but a 
decomposition on the basis of the cuts is certainly a good first approximation to the problem. 
Consequently, automatic cut recognition is a goal that many projects are pursuing (e.g., Zhang et 
al. 1995 [13]). While the automatic recognition of hard cuts is comparatively simple and can be 
considered largely solved, the distinction of soft cuts from rapid or complicated camera 
movements is much more complex. Even for this, however, reliable solutions in the next few 
years can be expected [13]. An initial classification of the discovered setting can be done by 
filtering out characteristics such as colors or certain regular shapes. Such "features" can then 
flow back into the cut recognition process (Hampapur et al. 1995 [6]). 

2. While the approaches just mentioned forgo an analysis of the film content. There are 
already ambitious projects that intend to automate the content-based classification or indexing of 
image information as well, relying on image recognition methods. It has been seen, however, that 
such systems will be available in the near future only for precisely delimited "domains" of image 
or film information (maps, for instance), because they presume a specific knowledge base 
(Pentland et al., 1995 [8]), in which shapes or movements are available in abstract form. It is also 
scarcely conceivable that moods or associations that images convey and which are also "worthy 
of documentation" will ever be detectable in this manner. 

3. More promising in the short to medium term is the attempt to use information 
accompanying the image information such as the audio narration (Gauch et al., 1994 [5]; Smith 
et al, 1995 [9]) or subtitles for automatic indexing. Of course, only one content aspect of many 
can be acquired in this way, but for an initial verification of the material in the archive this is 
certainly sufficient. Information obtained in this way can naturally also be further refined with 
automatically obtained "features" (Srihari, 1995 [10]). A deficit of this approach is that objects 
and actions that are only indirectly visible in the film can scarcely be detected in this manner 
because this accompanying information as a rule supplies more an interpretation than a 
description of the image content. Overcoming this deficit is a central concern of AMPHORE. 

4. Because of the multiple levels of meaning in image material, no indexing can be 
complete, which is why browser tools for examining the archive are much more important for 
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image and film databases than for text databases. These tools offer various ways of moving from 
one object in the holdings to the next: by following explicitly placed reference (hyperlinks), by 
(possibly implicit) placement of database queries that lead from the present unit to units with 
similar content or by implicitly searching for visually similar images (which can be achieved, for 
instance, by the comparison of "features"). While such a tool is practical for an image database, 
an additional problem in the case of film databases is that a film sequence can scarcely be 
surveyed "with one look" and therefore a relatively large amount of film material cannot be 
searched rapidly without additional technical support. Alongside the traditional techniques of fast 
forward and reverse, a digitization of the film offers the possibility of automatically determining 
a still image representing the shot (Arman et al., 1994 [1]) and accessing it directly. Particularly 
attractive here is the attempt to generate a new artificial image from all the images of a shot that 
shows the entire space covered by the camera in a scene at a single glance (Teodosio/Bender, 
1993 [12]). 

5. Even if all the aforementioned approaches are developed to perfection, the 
documentation process will still not be fully automatable; in media documentation, moods and 
associations that are based on human life experience play a great part. 3 It is therefore obvious to 
provide targeted support for this work process — and that is a concern in AMPHORE. Thus, for 
instance, many institutions work in multiple phases: new acquisitions are covered only roughly 
in a first step, so as to have at least some verification, and only later is a more detailed content 
description undertaken. Depending on the estimated importance of the material, these later steps 
may also be omitted. An undesired effect brought about by manual indexing 4 is the inter-indexer 
effect, which is particularly strong in the case of film documentation (see below). In 
AMPHORE, there is an attempt to minimize this effect, which can occur even inside a film in 
case of sequence-precise documentation, by special support. 

To bring it all down to a simple formulation, it is intended with AMPHORE, using 
hypermedia (= hypertext + multimedia) methods, that film sequences will be documented as 
precisely as possible and will become searchable. This implies in detail: 

Film material, accompanying material and documentary information form a unit, i.e., 
they are to be linked together in terms of content and also technically. In AMPHORE, this is 
achieved by providing the entire film material in digital form. This is currently being done by 
JPEG compression (Motion JPEG); in the next stage, that will be replaced by the more suitable 
MPEG compression. The quality that is achieved is certainly far from the original quality in 
every case, but is completely sufficient for examining the content of the material. 

Cross references between information components are supported implicitly and 
explicitly. "Implicitly" means in this case that similar sequences can be gathered together with 
the aid of suitable database queries. "Explicit" support is understood to mean links and references 
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that do not follow from the implicit linking or that are to be particularly emphasized. Thus, for 
instance, a text passage in the accompanying material can contain a link to an explanatory or 
associated film sequence. This link is processed in AMPHORE and can then be traced 
interactively (i.e., by clicking with the mouse). 

A number of problems occur in relation to content-based sequence documentation. On 
the one hand, a dilemma known from image documentation also recurs in film documentation; 
the phrase "a picture says more than a thousand words" summarizes this complex of problems. 
There is an attempt in AMPHORE to meet this problem in two ways. First, it is attempted by 
means of a rule corpus to separate directly perceivable "phenomena" from interpretive statements 
on the material (this corresponds roughly to the separation between "ofhess" and "aboutness"); 
second, documentation is strictly controlled by thesauri. It is evident that the problem is not 
eliminated by these conventional means, but is at best ameliorated. Thus, a case is illustrated in 
[3] in which a group of 18 test subjects (not only documentarists) only twelve persons used the 
term "Eiffel Tower" for a picture that shows nothing but the Eiffel Tower and the Seine (used 
nine times) in the foreground. 

If, as in AMPHORE, film sequences are documented instead of individual images, then 
additional questions arise. Two items are worthy of particular mention. First the question arises 
as to who actually defines the scope of a film sequence, and thus that of a documentary unit. In 
AMPHORE, this is the documentarist himself, which is why software tools are made available to 110 
determine or change sequence boundaries. The fact that the indexer established the units of 
indexing, in contrast to image documentation, naturally amplifies the inter-indexer effect 
considerably. That results in new requirements on a rule corpus. 

A second difference between film and image documentation is that for film indexing the 
action moves more into the foreground with respect to the objects represented. This can be met 
by substantivizing verbs ("seizure," "fox," "rabbit"), which does not always appear precise 
enough. From the description ("bird flight," "magpie," "crow"), for instance, it is ultimately 
unclear which bird is flying here, the magpie, the crow, both or some other, unspecified bird. 
Consequently indexing in AMPHORE is syntactic, with a subject field, which must be filled out, 
an optional verb field and up to four optional object fields. 5 Subject and object fields are 
controlled by a shared "noun" thesaurus and the verb field by its own "verb" thesaurus. 

At least one such "SPO" construct must be assigned for sequences, which means that a 
single subject is sufficient for description, since predicate and objects are optional. As a rule it 
will usually be desired to describe the material more precisely, which is why it is also possible to 
assign several constructs to the same film sequence. In this case, however, the order of the 
constructs reflects the weight of the underlying statement or action. Thus, the first construct is 
supposed to describe he central action of the sequence. With the aid of the formalism, the above 
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descriptions become ("fox," "seize," "rabbit") and ( "magpie," "fly") alongside ("crow," "fly"), 
which makes it clear that both birds fly in the scene, but the flight of the magpie is more visually 
prominent. 

In searching for film sequences in AMPHORE, a search via this syntactic indexing 
frequently supplies results, but by no means always sufficiently accurate ones. For this reason, 
all the other information belonging to the film in AMPHORE must also be present and 
searchable. In detail, these are: 

• Formal data on the film in consideration such as director, title, production site(s), 
production year(s) and also a brief note on the content (frequently present). 

• Accompanying information such as script, commentaries, scientific monographs 
(especially for scientific films), etc. 

• Voice-overs, dialog, if available as text. 

• Technical information such as the length of the individual sequences, their quality (e.g., 
black-and-white or color shooting) or cinematographic features (night shot, close-up, 
slow-motion/time lapse, etc.). 

• The film material itself in digital form and 

• the digitized audio material, if available separately from the image material. 

If all these information components are put together, a quite complexly structured 
mixture of descriptors, freely composed texts, numerical and multimedia data results. A suitable 
model is required to be able to model this complex syntactically. For a number of reasons, 
SGML (Structured Generalized Markup Language) was chosen for AMPHORE: 

• SGML is a widely accepted ISO standard (ISO 8879), expressed particularly in the fact 
that recently a considerable supply of software supporting SGML has been developed for the 
market. 

• In particular, document management systems that explicitly support SGML have also 
been available for two or three years. Here it must be noted critically, however, that products are 
generally still battling with "infantile diseases," which have a negative effect on their stability 
and performance. 

• The HTML format (Hypertext Markup Language) which is widely used on the Internet 
conforms to SGML, which means that a document management system based on SGML can also 
be used, at least in parts, for management of HTML documents. 

For any use of SGML, the development of a DTD (Document Type Definition), or the 
selection of an already existing DTD, in which the concrete document structure is established, is 
fundamental. In the case of AMPHORE, a new development was inevitable. The AMPHORE 
DTD specifies that, for every film, a document is created that consists of one mandatory and four 
optional parts, namely: 
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• the DOKUMENT element: here the "filmographic" information is contained, i.e., 
author, title, etc. This part serves for verification of the entire film in the database and is 
therefore the only mandatory part of the AMPHORE document. 

• the FELMDAT element: it contains a link to the file with the film data. The multimedia 
information is thus not stored directly in the database; instead, links to this information are 
managed. As long as the consistency of these links is guaranteed, no technical or methodological 
drawback results. 

• the AUDITEXT element: it contains the dialog or narration of the film, if this 
information is available in text form. This element is downwards-compatible with HTML; i.e., it 

can be displayed without conversion in HTML browsers like Netscape or Mosaic. 11 1 

• the BEGLEIT element: it contains the accompanying information as outlined above 
and/or links to it. This part is also downwards-compatible with HTML. 

• the SEQUENZ element: it contains arbitrarily many elements of types SEQ or FRAME, 
which describe individual images of the film by means of the above-mentioned SPO constructs, 
by free text downwards-compatible with HTML and by technical data serving on the one hand to 
control the presentation of the films and on the other for searching. 



CD 



UNIX 
SCML- 
Volltexc 
Datcnbank 



Document 



SGML- 
Dokumeme 



Internet 
(RPOFTP) 



SGML* - 2l4£ - 
Dokumeme 



Dok u mental ions - 
Werkzcuc 



(D 

Antra gen 



Recherche- 
Werkreug 



Figure 1 : Client-server architecture of the AMPHORE system 
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AMPHORE is conceived as a multiseat system with client-server architecture. Situated in 
the center as the server is a UNIX database computer, which is accessed by PC workstations via 
Internet protocols (Remote Procedure Call RPC and File Transfer Protocol FTP). Documentation 
and Search tools can be called as needed at the PCs. Cf. Figure 1 . 

A technical problem of central importance is the digital storage of the films. Modern-day 
databases do permit the definition of correspondingly large objects (so-called BLOBs - Binary 
Large Objects), but this approach is ruled out [text possibly omitted from original] because the 
database server cannot guarantee that the client will be supplied with a continuous data stream, 
which it needs to play back the film "cleanly,' 1 i.e., without hesitation. An alternative would be to 
load the film data completely into the client computer before playback, which would in turn 
result in this loading time being longer than the actual film; this is an approach that is common in 
the Worldwide Web, but is obviously unsuited to "interactive" film documentation. 

In AMPHORE it is therefore assumed that the film data of a film are represented in a file 
of their own, while only the name of this file is managed in the database. The format in which 
this data exists and the technology with which the files are accessed are also not further specified 
in AMPHORE. Thereby the system is open to various solutions regarding the costs of the 
infrastructure, the scope of the film material in terms of time and its quality. An economical 
solution might look like this: all films are present on CD-ROM ( a CD-ROM holds roughly an 
hour of film including stereo audio, assuming the customary MPEG-1 format) and this number 
of CDs are available at the documentarist's workstation. Inserting the respective CD in [text 
missing in original] ... For smaller archives with up to 1,000 hours of material this approach is 
well manageable and often probably the only realistic one for reasons of cost. 

A more expensive solution, which can also gradually emerge from the previously 
sketched simple solution, is to manage the optical data media in jukeboxes (such media need 
necessarily be the CDs customary today; new "multimedia CDs" with ten to twenty times more 
capacity will appear on the market before the end of the millennium). The selection of the 
appropriate data carrier would then take place automatically; a problem not to be underestimated 
here is the necessity of transferring the data from the jukebox to the workstation. For this, a 
sufficiently fast network, another investment in some circumstances, is necessary. 
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Figure 2: Overview window for calling the five subeditors for editing a . . . [text omitted from 



original]. 
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3. The documentation tool 

Viewed technically, the AMPHORE documentation tool is an editor for editing SGML 
documents that match the AMPHORE DTD. If such a document is loaded, then this is 
represented by a menu (cf. Figure 2) that permits the editing of the five aforementioned 
document parts. 6 

Accordingly, there are five subeditors that can be called up via the menu items. As an 112 
example, Figure 3 shows the subeditor for the DOKUMENT element. Since the use of these 
subeditors can bring about a number of open windows and, moreover, several documents can be 
edited simultaneously, confusing situations can easily arise, which is countered in AMPHORE 
by two means (see Figure 4). 

On the one hand, comfort functions are provided that make it possible to close all or 
certain subeditors, to iconify them or to bring them to the front. On the other hand, subeditors 
that belong to the same document all have the same color. Thus, up to six documents are 
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differently colored, the colors being chosen such that, from the perspective of color theory, 
maximum contrasts result. Tests have shown that even users who are "poor with colors 11 manage 
these well. 
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Figure 3: AMPHORE form for acquiring the general film-describing data 
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Summary 

The egg-laying of the muddy darters dragonfly (S. sanguineum) and probably that 
of the black meadow hawk dragonfly (S. danae) in tandem flight is shown. At a 
photographing frequency of 500 fps, details of the male-female interaction as well 
as those of the wing movements, become recognizable. A male carries a 
three-winged female to lay eggs. 
Film description 
Description list 




Figure 4: Working situation during documentation with AMPHORE 
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4. The Search Tool 

The basic idea of the AMPHORE search system 7 is to make use of all, or as closely as 
possible to all, of the information available in AMPHORE for the search. This is achieved by 
offering the user four different search forms that can be combined in arbitrary numbers with the 
operators "AND" and "OR." In detail, these are: 

• The DESCRIPTORS form (Figure 5), which permits the search via SPO constructs. 
The individual fields are automatically checked for thesaurus conformity before the start of the 
search, i.e., "wrong" queries are not possible. With the aid of a thesaurus browser (call for the 
individual fields via the respective "TH" switch) the query fields can also be filled in by menu 
control, which is of use in practice particularly to the occasional user of the system. /73 
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Figure 5: Form for search with descriptors or subject-predicate-object constructs 
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* • The FULL TEXT form (Figure 6). All free text parts of the documents are indexed 
taking into account a stop word list and are offered as a menu (to the left in the figure). The full 
text search can relate to the entire document or only to the checked-off parts (to the right in the 
figure). 
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Figure 6: Form for full text search in AMPHORE 
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• The DOCUMENT form (Figure 7) is a traditional search form for the film database. 
This is how films of a given director or with a given title, etc. are found. 
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Figure 7: Form for searching for films by means of formal film database [illegible] 
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• The TECHNIQUE form (Figure 8) ought to be of particular interest if the search is 
carried out for production support. With it, one can find, for instance, sequences of a given 
duration or with given characteristics (e.g., black-and-white shots or slow-motion sequences) that 
are not content-specific. The typical numerical data here need not be exact, but can also be listed 
as an interval, e.g., "120 sec ± 30 sec" for sequence length. 
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Figure 8: Form for searching for film sequences based on technical features 
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5. Outlook 

AMPHORE can be viewed as a "frame" system that offers space for many expansions 
and applications. Consider merely the fact that audio and (still) images are merely special cases 
of sound film from the technical perspective and therefore can be stored and managed no 
differently than other films. Of course, experiences with such integrated media archives show 
that descriptions independent of information type must be more precisely separated from content 
for this purpose than has previously been the case with the AMPHORE DTD [11], 

In regard to technologies of image processing, such as the recognition of cuts, much less 
objects, they can be used to support the documentation work and thus considerably increase 
work efficiency. It should also be noted that these techniques do not constitute a restriction of 
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"documentary freedom." Background knowledge and experience of the world will continue to 
make people indispensable in media documentation for the foreseeable future. 



Film documentation; SGML application; system; searching 



Literature 



(1| Arman. F.; Depommier. R.; Hsu. A.; 
Chiu. M.-Y.: Content-Based Browsing of 
Video Sequences. In Proc Second ACM 
International Conference on Multime- 
dia. San Francisco. CA, October 1994. 
S. 97-103 

[2J Carlson. C: Perspectives of a Hyper- 
media Film Sequence Database. In Innova- 

In: Innovative Medienanwendungen. Beitrage der IAMS/GMW-Tagung [Innovative 

media applications: Contributions of the IAMS/GMW convention]. Gottingen. Medien in 

der Wissenschaft [Media in Science], Vol 2. Gottingen, 1993, pp. 92-97. 

(3) Enser. P. G. B.: Pictorial Information 
Retrieval, (n Journal of Documentation. 
Vol. 51. No. 2, June 1995. S. 126^170 



[4] Fischer, A.: Entwurf und Implementation eines theasurusunterstutzten Suchwerkzeugs fUr 
eine Videodatenbank [Design and implementation of a thesaurus-supported search tool 
for a video database]. Thesis, Fachhochschule Darmstadt, Computer Science Department, 
March, 1995. 



(5} Guuch S.; Aust. R.; Evans. J.; 
Couch /; Minderu C: Niehaus. D.; Ro- 
berts. J.: The Digital Video Library System: 
Vision and Design. In Proc. Digital Libra- 
ries '94. June I994 t S. 47-52 

(61 Hampapur, A.: Jain. R.; Weymouth. 
T. E.: Feature Based Digital Video Inde- 
xing. In Proc. IFIP Third Working Confe- 
rence on Visual Database Systems ( VDB-3). 
Lausanne, March 1995. S. 99-121 



18 



[7] Miiller, A. Entwurf und Implementation eines Werkzeugs zur Dokumentation von 

Videoclips [Design and implementation of a tool for the documentation of video clips]. 
Thesis, Fachhochschule Darmstadt, Computer Science Department, March, 1995. 



(8| Pentland. A.; Picard. R.; Davenport 
C; Haase. K.: Video and Image Semantics: 
Advanced Tools for Telecommunications. 
In Proc Electronic Publishing and the 

Information Superhighway. Boston MA. 
May 30 - June 2, 1995. S. 47-52 

(9| Smith. M. A.: Chnsiet. M. G.: Automa- 
ting the Creation ot a Digital Video Library. 
In Proc Third ACM International Confe- 
rence on Multimedia, San Francisco. CA. 
November 1995, S. 357f. 

[ I0| Srihari. R. K.: Automatic Indexing and 
Content- Based Retrieval of Captioned 
Images. In IEEE Computer* September 
1995. S. 49-56 

III) SUIIow. K.; Gabel-Becker. I.; Ocken- 
feld. M.; Puxz. W.; Roth. G.: MultiMedia 
Forum: an interactive online journal, in 
Electronic Publishing, Vol. 6, No. 4. 
December 1993, S. 413-422 

(12] Teodosio. L; Bender, W.. Salient Vi- 
deo Stills: Content and Context Preserved. 
In Proc. First ACM International Conferen- 
ce on Multimedia. Anaheim, CA, Ausum 
1993. S. 39-46 

113] Zhang. H; Low, C Y.; Smoliar. S. W : 
Video Parsing and Browsing Using Com- 
pressed Data. In Multimedia Tools and 
Applications. Vol. I, No. I. March 1995. 
S. 89-111 



19 



Notes 



1 These may well be viewed as "formal" because they are not directly content-related. One 
should consider, however, that the content structure is reflected in these data. 

2 It should not go unmentioned that such innovations will in the medium term reduce personnel 
expenses in those achieves that already index to the sequence-level. It is equally clear that this 
effect is compensated for in the smaller achieves by the additional value creation that is then 
possible. 

3 And acquire even more significance with the increasing automation of work and everyday life. 

4 And, incidentally, by automatic indexing as well, if different systems are involved. 

5 This approach originates with C. Carlson [2], 

6 An idea from A. Miiller, whose paper [7] described the technical details of the AMPHORE 
"editor." 

7 Technical details are found in [4]. 



